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The Data Quality Campaign (DQC) is a national, collaborative effort to encourage and 
support state policymakers to improve the collection, availability and use of high-quality 
education data and to implement state longitudinal data systems to improve student 
achievement. The campaign aims to provide tools and resources that will assist state 
development of quality longitudinal data systems, while also providing a national forum for 
reducing duplication of effort and promoting greater coordination and consensus among the 
organizations focusing on improving data quality, access and use. 


To help states learn from one another, DQC staff visited state education agencies which are 
engaged in different stages of development of their longitudinal data systems. In 2006, staff 
visited Florida, Utah, Virginia and Wisconsin; 2007 meetings were held in Massachusetts 
and South Carolina. The DQC will continue to highlight specific state efforts to develop and 
use longitudinal data systems, and will conduct additional case studies in 2008. 


This publication was produced by the Data Quality Campaign/National Center for 
Educational Accountability for submission under contract with the Bill & Melinda Gates 
Foundation. The report was written by Cherry Kugle, Ph.D. and Nancy Smith, Ph.D. and was 
reviewed and approved by the Massachusetts Department of Education. 


© Copyright 2007. Data Quality Campaign. All rights reserved. 
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Massachusetts: Connecting the Dots 

The Massachusetts Department of Education (MDOE) oversees 389 school districts and 59 
charters, encompassing over 1,800 schools in which over 960,000 students were enrolled in 
the 2006-07 school year. 

As of the 2007 Data Quality Campaign (DQC) annual survey of state longitudinal data 
systems, Massachusetts has implemented eight out often essential elements. The only 
elements the state has not implemented include the ability to connect teacher and student 
data together and a system to track which courses students take along with their 
completion status (e.g., grade or pass/fail status) in those classes. 

History 

In 1993, the Massachusetts Education Reform Act (MERA) was enacted, leading to the 
development of statewide learning standards (the Curriculum Frameworks) and the 
Massachusetts Comprehensive Assessment System (MCAS). 

Development of the statewide Student Information Management System (SIMS) was begun 
in 1998-99 with the first data collection in fall of 2001. SIMS was a direct result of MERA 
and provides a secure portal for districts to transmit data to the MDOE on individual 
students, using unique student identifiers. SIMS data elements are added and refined as 
reporting requirements evolve. 

Teacher Information Systems: In 2002, the state launched the Educator Licensure and 
Recruitment System (ELAR), a web- based system that allows current and prospective 
Massachusetts educators to complete most licensure- related transactions on the Internet. 
The Educator Personnel Information Management System (EPIMS) is in the pilot stage, with 
statewide implementation currently planned for Fall 2007. EPIMS will replace the aggregate 
data collection on personnel currently employed in public education with individual educator 
information. EPIMS includes state-assigned educator identification numbers, multiple 
descriptors including the Highly Qualified status required under No Child Left Behind (NCLB), 
and a protocol to collect, define and link educators to the roles they perform and classes 
they teach. EPIMS incorporates the National Center for Education Statistics (NCES) course 
codes, with districts mapping courses taught to the NCES codes. 

State Data Warehouse: MDOE maintains a database of itemized student responses to the 
MCAS and Massachusetts English Proficiency Assessment (MEPA). An Enterprise wide 
Education Data Warehouse, in the pilot stage, links individual student data to MCAS and 
MEPA assessment data. Plans call for the data warehouse to provide the infrastructure for 
teachers, administrators and policy leaders to be able to combine and analyze datasets that 
are not currently connected. 

An additional ongoing effort involves a SIMS data exchange with the state's Board of Higher 
Education data warehouse to populate a K-16 database and ultimately, PK-16. The 
Department of Education has provided data on four graduating classes to the Board of 
Higher Education that has been matched to student records at public higher education 
institutions. The resulting analyses were recently shared with policy makers to highlight the 
relationships between high school and college performance. To date, data has only been 
shared from K-12 to higher education; the K-12 data system has not received data back 
from higher education. 
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Current Developments 

In FY08, the state legislature provided, and new administration supported, $5.2M in the 
state budget for Information Services and Technology/ This is a significant increase from the 
$700,000 received in FY07. 

In March 2007 the MDOE applied for, but did not receive, a grant from the Institute of 
Education Sciences (IES) of the United Stated Department of Education (USDE) to complete 
the foundation for the Massachusetts Education Data System for Improving Student Success 
(MA-EDSISS). The objectives of this initiative are to: 

• link students in SIMS to their assigned classes and then to EPI MS to create a 
complete student (including performance) to class to teacher connection 

• complete critical functionality in ELAR (educator licensure), including incorporating 
educator item responses on the Massachusetts Tests for Educator Licensure (MTEL) 

• fully link ELAR (educator licensure) to EPI MS (educator employment) to improve the 
accountability and effectiveness of the licensure procedures and enable districts to 
ensure that licenses of employed educators are current and course assignments are 
appropriate 

• assign educator identifiers to individuals at the beginning of their career path to 
enable research leading to more informed policy decisions on educator recruitment, 
preparation, development, mobility and retention 

• coordinate and improve data collections by streamlining procedures, increasing 
compatibility, and strengthening business rules to decrease data collection burden 
and increase the efficiency and validity of all data collection systems 

• conduct a feasibility study on moving from periodic data collections to a more 
continuous data stream and, consequently, more timely information 

Since MA was not awarded one of the IES grants, there are three key areas that are at risk 
of being delayed or not completed: 

1. Development of a course completion system for students with the ability to 
connect student and teacher data; 

2. Staff identifier - the goal was to assign a staff identifier (ID) as soon as 
people enrolled in a teacher preparation program and/or upon taking the 
licensure test, but the ID will continue to be assigned when the teacher joins 
the district; and 

3. MDOE will be unable to conduct a feasibility study for implementing the 
Schools Interoperability Framework Association standards. 


Implementation Challenges - staffing and funding 

Almost all of the development and maintenance work for these data systems is done in- 
house by the Information Services (IS) and Information Technology (IT) staff, since 
resources are generally not available to hire corporate vendors. The ability of the staff to 
produce useful systems and reports is substantially enhanced when project managers are 
available to oversee the work. 

IS staff currently spend the majority of their time on federal mandates, with the remainder 
spent on ad hoc requests. The ability of the department to analyze and report the data on 
behalf of their stakeholders is hindered by the volume of the ad hoc requests, a situation 
that is anticipated to be eased by the enterprise education data warehouse and staffing a 
research and evaluation office within the MDOE. 
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The majority of EPI MS development has been accomplished with funds provided by state 
technology bonds. Although Bond money was stalled for much of FY07, additional Bond 
funds were provided to continue the statewide implementation of EPI MS. 

Although there has been transition at the Governor and Commissioner level in MA, 
leadership within the MDOE is committed to the continued development of the data 
systems. 


Costs of Building, Managing and Updating Massachusetts' Longitudinal 
Data System 

Estimated costs to the state: As always, the answer to the question of costs is a 
complicated one. 

• The I S/IT department consists of 60+ staff. Although estimated costs of $4 million a 
year are incurred by the department, the allocated operating budget over the past 
five fiscal years was approximately $700,000 a year. To make up the shortfall, other 
MDOE program areas and sections are charged for work done on their behalf by 

I S/IT. This practice covers some costs, but it takes away from time on I S/IT-specific 
projects which adds another cost. As mentioned previously, IS&T was appropriated 
$5.2M in the state budget for FY08.For SIMS, the state currently has a contract with 
one consultant to oversee the implementation of changes to the data elements. The 
costs of maintaining the infrastructure of the system are separate. The costs of 
technical assistance to districts during the change process are spread out over the 
staff, as are the variety of other tasks associated with this effort. 

• EPI MS development was begun with approximately $1 million in funds generated 
through IT bonds. An additional $1 million has been provided to complete the 
development and support statewide implementation. A portion of the EPI MS effort is 
supported by a grant from the National Governors Association (NGA). 

• The data warehouse initiative was started with a competitive bid on a pilot project in 
early 2005. After exploring the experiences of other states and seeking a proof of 
concept for use by the department, MDOE selected Cognos as the vendor. The state 
legislature subsequently provided $5.5 million for this effort. This amount paid for a 
statewide license for the state department, all district and school staff, and policy 
makers, and the remainder supported three contractors and two state employees. 
The department plans to pay for maintenance of the system. Districts are expected 
to cover their own costs for training, consulting and professional development. 
Competitive two-year Title II, Part D grants (Enhancing Education Through 
Technology or Ed Tech grants) from the USDE have been targeted to districts that 
are interested and ready to participate in the data warehouse initiative. 

• The state purchased a right- to- use license for districts to have access to assessment 
data (via TestWiz, a product of Datametrics). The cost, including training, is 
approximately $400,000 a year. 

Estimated costs to districts: 

Staff in two districts of varying sizes were interviewed for this report. Both agreed that it is 
hard to estimate costs associated with improvements in the state data system. 

• Many costs are absorbed by the district, particularly personnel costs, or are 
sometimes absorbed by the district's vendor. 

• One large district estimated that they had 2 human resources staff and one data 
director each spend about 150 hours on work related to the EPI MS system. 
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• Annual costs for service contracts and maintenance contracts in that district run 
approximately $500,000. 


Comments from districts 

• Continue to involve districts in planning and pilot phases of development. Seek 
involvement of a variety of districts (in terms of size, geography, technology, etc.) 

• If possible, provide even $5,000-10,000 to each district for human resources, 
hardware or software expenses. While it is only a small amount of the cost to a 
district, it will help things move forward and promote goodwill. 

• Allow plenty of time for districts and districts' vendors to make changes to data 
systems prior to state data collections. 

• Provide training and ongoing communication to district and school personnel in the 
form of online materials, in-person training, Frequently Asked Questions (FAQs), and 
ongoing email announcements about upcoming changes and interim solutions. 

• Completion of the data warehouse will help districts in many ways, especially in 
getting data back in more useable formats. In the current system, districts indicate 
that it is not always easy to get data back from the state. 

• Provide training throughout MDOE about data issues and standards so that districts 
will get the same response regardless of whom they ask questions. 


Lessons Learned 

• "Any time you attach money to data or make it public, you will get more attention 
paid to obtaining quality data." 

• With aggregate data, inaccuracies cannot be detected as early or clearly as with 
individual student level data. 

• The less one is able to invest in project management, the harder it is to move the 
project forward. 

• It is important for states involved in similar efforts to anticipate the maintenance and 
continued development costs after the major initial development effort is 
accomplished. It is vital that the necessary resources be available to keep the data 
systems functional and evolving. 


Recommendations for Future Actions by MDOE 

• Engage a data advocate to promote the need for development and maintenance 
among state policymakers (e.g., Chief, Governor's office, legislators) - perhaps a 
former commissioner or deputy commissioner, as in other states. 

• Continue to involve a variety of districts in design and pilot phases of new projects. 
Ensure a variety of voices in terms of district size and location, but also in terms of 
the types of individuals involved. For example, in the design of reporting and 
analysis tools associate with a data warehouse, include district- level research staff as 
well as IT staff to ensure that both the analytical and technological perspectives are 
addressed. 

• Develop a communications/marketing plans (with help enlisted from a state data 
advocate or external sources) to educate state and district policymakers and other 
MDOE staff of the importance of cohesive data systems (P-20), professional 
development on the use of data, and the need to move from thinking of data only in 
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terms of meeting state and federal reporting needs to using data to inform state and 
local policy and funding decisions. 

• Hire project managers to oversee the development of new functional areas of the 
data system (i.e., EPIMS, ELAR and data warehouse) or for major enhancements to 
existing systems. Designing, developing, and deploying an extensive statewide 
longitudinal data system cannot be accomplished by adding these additional 
responsibilities to existing staff. Oversight of the many committees, involvement in 
planning meetings, work with vendors and other project management activities that 
must be accomplished in-house is not a part-time prospect. 

• Develop a development plan for gathering financial resources, supplemental to state 
funds, which will ensure that costs associated with maintenance and annual licenses 
and contracts will be covered to ensure that the data system is sustained at a state- 
of-the-art level, and especially to continue work on MA-EDSISS in lieu of not 
receiving an I ES Statewide Longitudinal Data Systems grant. 
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